{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --user folium"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab 14 - Mapping data\n",
    "\n",
    "In this lab, we will use the [folium package](https://python-visualization.github.io/folium/quickstart.html) to create maps with markers on them and to make [chloropleth maps](https://en.wikipedia.org/wiki/Choropleth_map).\n",
    "\n",
    "First import folium and pandas."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import folium\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create a folium map, we just need to provide a latitude and longitude.  Latitude specifies the north-south position on the globe, with north of the equator being positive and south of the equator being negative.  Longitude specifies the east-west position on the globe, relative to the meridian at Greenwich, UK."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folium.Map(location=[40.8747, -73.8951])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also save the map as a variable and display it that way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m = folium.Map(location=[40.8747, -73.8951])\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's put a marker on the map at Lehman College, which is located at 40.8733° N, 73.8941° W."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folium.Marker([40.8747, -73.8951], \n",
    "              popup=\"Lehman College\", \n",
    "              tooltip=\"Click me!\").add_to(m)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try clicking on the marker.  What does the parameter popup does?  What does the parameter tooltip does?\n",
    "\n",
    "Add another marker for City College, which is located at 40.8200° N, 73.9493° W.  Use either the popup or tooltip parameter (your choice) to label the marker as City College."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also change the color and image on the marker.  For example, the following code marks the location of Hostos Community College with a red marker with an info sign on it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folium.Marker([40.8181, -73.9269], \n",
    "              popup='Hostos Community College', \n",
    "              tooltip=\"Click for info\",\n",
    "              icon=folium.Icon(color='red', icon='info-sign')\n",
    "             ).add_to(m)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can usually find the lattitude and longitude coordinates for many New York landmarks by using the landmark name and the word coordinates in the same search.  Find the coordinates for another landmark in New York and add that marker to your map.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Location of Recycling Bins\n",
    "\n",
    "We'll now create a map of all public recycling bins in New York City, which is available on Open Data NYC.\n",
    "\n",
    "- the dataset of all public recycling bins in New York city is [here](https://data.cityofnewyork.us/Environment/Public-Recycling-Bins/sxx4-xhzg).  To download: \n",
    "    - click \"View Data\" (blue button in upper right)\n",
    "    - on the next page, click \"Export\" (in menu in upper right)\n",
    "    - click \"CSV\" to download\n",
    "    \n",
    "Read the CSV file into a dataframe called `bins`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There is some missing data, so let's drop all rows with an NaN value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bins = bins.dropna()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a new folium map variable called `bins_map` centered at 40.7128° N, 74.0060° W (New York City coordinates)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Recall that in a loop, the counter counts from 0 to the range - 1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(5):\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use this to loop through the rows in our dataframe `bins`.  Below we loop through the first 10 rows of `bins`, store the current row in the variable `row`, and use it to print the latitude and longitude."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(10):\n",
    "    row = bins.iloc[i]\n",
    "    print(\"Coordinates: \", row[\"Latitude\"], row[\"Longitude\"] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's plot the bins on the map.  We want to loop through all the rows (how can we find the number of rows?) and use the latitude and longitude from each row to create a new Marker for our map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(bins.shape[0]):\n",
    "    row = bins.iloc[i]\n",
    "    folium.Marker([row[\"Latitude\"], row[\"Longitude\"]],\n",
    "                 tooltip = row[\"Park/Site Name\"]\n",
    "                 ).add_to(bins_map)\n",
    "bins_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What's the closest bin to Lehman College?\n",
    "\n",
    "There's a column in the `bins` dataframe called \"Park\\Site Name\".  Can you re-plot the map of the bins, but using the value in this column as the tooltip text?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Can you create a new map with the locations of all the CUNY colleges?  The coordinates are available at https://data.ny.gov/Education/City-University-of-New-York-CUNY-University-Campus/i5b5-imzn/data.  Click \"Export\", then \"CSV\" to download as a CSV file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chloropleth Maps\n",
    "\n",
    "A chloropleth map is a map with areas shaded or colored in proportion to the mean (or some other statistical variable) of some property of that area (like income, population density, etc.)\n",
    "\n",
    "For our first choropleth map, we'll use the NYC school district boundaries available from\n",
    "Open Data NYC Planning:\n",
    "    - go to \\https://www1.nyc.gov/site/planning/data-maps/open-data/districts-download-metadata.page\n",
    "    - scroll down to \"School, Police, Health & Fire\"\n",
    "    - in the school district row, click the geoJSON button to download in that format.\n",
    "    - save the file as nyc_school_district.json if necessary\n",
    "    - upload to Jupyter Hub\n",
    "\n",
    "The district math scores are at: https://infohub.nyced.org/reports-and-policies/citywide-information-and-data/test-results\n",
    "    - scroll down to Math Test Results 2013 to 2019\n",
    "    - download the district Excel file\n",
    "    - open the Excel file and export the page with all the scores as a CSV file\n",
    "    - OR:  download CSV file directly at http://comet.lehman.cuny.edu/owen/teaching/mat328/math_district.csv\n",
    "\n",
    "Create a map called `schoolMap`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a layer, shaded by test scores, and add it to your map:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "folium.Choropleth(geo_data =\"nyc_school_districts.json\",\n",
    "                     fill_opacity=0.5, line_opacity=0.5\n",
    "                     ).add_to(schoolMap)\n",
    "schoolMap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To save the map as an HTML file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "schoolMap.save(outfile='testScores.html')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's color the districts by the mean 8th grade math score in 2018.  Read the math district scores CSV file into a dataframe called `fullData`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter this dataset to only include scores from 2018 and 8th grade.  Call your new dataframe `scores8th20181`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To add the shading to the map:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Create a layer, shaded by test scores:\n",
    "folium.Choropleth(geo_data =\"nyc_school_districts.json\",\n",
    "                      fill_color='YlGn',\n",
    "                      threshold_scale = [100,200,300,400],\n",
    "                     data = scores8th2018,\n",
    "                     key_on='feature.properties.SchoolDist',\n",
    "                     columns = ['district', 'Mean Scale Score'],\n",
    "                     fill_opacity=0.5, line_opacity=0.5\n",
    "                     ).add_to(schoolMap)\n",
    "schoolMap\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}